Development and Testing of Step, Error, and Event Frameworks to Evaluate Technical Performance in Peripheral Endovascular Interventions.

OBJECTIVE: Tools for endovascular performance assessment are necessary in competency based education. This study aimed to develop and test a detailed analysis tool to assess steps, errors, and events in peripheral endovascular interventions (PVI). METHODS: A modified Delphi consensus was used to identify steps, errors, and events in iliac-femoral-popliteal endovascular interventions. International experts in vascular surgery, interventional radiology, cardiology, and angiology were identified, based on their scientific track record. In an initial open ended survey round, experts volunteered a comprehensive list of steps, errors, and events. The items were then rated on a five point Likert scale until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) and > 70% expert agreement. An experienced endovascular surgeon applied the finalised frameworks on 10 previously videorecorded elective PVI cases. RESULTS: The expert consensus panel was formed by 28 of 98 invited proceduralists, consisting of three angiologists, seven interventional radiologists, five cardiologists, and 13 vascular surgeons, with 29% from North America and 71% from Europe. The Delphi process was completed after three rounds (Cronbach's alpha; αsteps = 0.79; αerrors = 0.90; αevents = 0.90), with 15, 26, and 18 items included in the final step (73 - 100% agreement), error (73 - 100% agreement), and event (73 - 100% agreement) frameworks, respectively. The median rating time per case was 4.3 hours (interquartile range [IQR] 3.2, 5 hours). A median of 55 steps (IQR 40, 67), 27 errors (IQR 21, 49), and two events (IQR 1, 6) were identified per case. CONCLUSION: An evaluation tool for the procedural steps, errors, and events in iliac-femoral-popliteal endovascular procedures was developed through a modified Delphi consensus and applied to recorded intra-operative data to identify hazardous steps, common errors, and events. Procedural mastery may be promoted by using the frameworks to provide endovascular proceduralists with detailed technical performance feedback.

Development, Feasibility, and Knowledge Impact of a Massive Open Online Course on Radiation Safety: A Multicentre Prospective Cohort Study.

OBJECTIVE: This study reports the development of an innovative, interactive Massive Open Online Course (MOOC) teaching radiation safety principles in the vascular workplace, using stepwise e-learning with multiple choice question tests (MCQs), educational videos, and a serious game. The aim was to study the MOOC impact on radiation safety knowledge and assess its feasibility and acceptability. METHODS: An international multicentre prospective study included team members active in the hybrid operating room. The MOOC was offered voluntarily via a secure online learning platform. A standardised MCQ test (15 questions) assessed radiation safety knowledge pre- and post-course (range 0 - 100%). Acceptability and feasibility were tested via the previously validated, Evaluating e-learning system success (EESS) model, using five point Likert scales. RESULTS: In eight centres across four countries, 150 of 203 invited endovascular team members consented. Over a seven week study period, surgeons (28%, including vascular surgery trainees and consultants), nurses (27%, including scrub, circulating and anaesthetic nurses), anaesthetists (43%, including trainees and consultants), and radiographers (3%) participated. Of those, 67% completed the course. The average radiation knowledge improved by 22.8% (95% CI 19.5 - 26.0%; p < .001) after MOOC completion, from 48% to 71% (standard deviation [SD] 14 and 15% respectively), requiring a mean time investment of 169 minutes (SD 89 minutes). In centres with a radiographer, mean knowledge gain after MOOC completion was significantly smaller (14%, SD 19% vs. 24%, SD 16%, p = .036). The course was deemed feasible and acceptable according to the EESS model with a total mean score of 3.68/5. CONCLUSION: This newly developed, multimodal MOOC was deemed feasible and effective across multiple international centres. The MOOC significantly contributes to radiation safety education of the entire endovascular team, improving radiation safety knowledge. The course may optimise workplace radiation safety behaviour and therefore enhance team and patient safety.

Massed Training is Logistically Superior to Distributed Training in Acquiring Basic Endovascular Skills.

OBJECTIVE: A "PROficiency based StePwise Endovascular Curricular Training" (PROSPECT) has proven its superiority over traditional training in a randomised controlled trial to acquire basic endovascular skills outside theatre, but real life adherence is low. This study aimed to compare the original distributed training format, where trainees learn at their own pace, with a massed training format offering the same content within a limited time span while exempt from clinical duties. Secondly, long term skills retention was evaluated. METHODS: A multicentre, prospective study allocated participants to the distributed D-PROSPECT or to a massed, compact version (C-PROSPECT) based on logistics such as travel time, participant and instructor availability. A multiple choice question (MCQ) test (max. score 20) tested cognitive skills. Technical skills were assessed using a global rating scale (GRS) (max. score 55), examiner's checklist (max. score 85), and validated simulator metrics. Data were collected pre- and post-programme and at three, six, and 12 months after programme completion. RESULTS: Over four years and in two countries, D-PROSPECT was implemented in two centres and C-PROSPECT in three. A total of 22 participants completed D-PROSPECT with a 41% dropout rate, and 21 completed C-PROSPECT with 0% dropout rate. All participants showed significant improvement for all performance parameters after programme completion: MCQ test (median 14.5 vs. 18; p < .001), GRS (median 20 vs. 41; p < .001), examiner's checklist (median 49 vs. 78.5; p < .001), and simulation metrics (p < .001). Scores of C- or D-PROSPECT participants were not significantly different. No significant differences were seen between groups during the retention period. CONCLUSION: PROSPECT significantly improves the quality of simulated endovascular performances using a massed or distributed training format. A massed training format of PROSPECT may be preferred to decrease dropout during standardised training to obtain basic endovascular skills in existing surgical curricula.

Facilitators and Barriers to Implementation of Simulation Based Education in Vascular Surgery in Europe.

OBJECTIVE: This study explored the status and availability of simulation based education (SBE) for learning vascular surgical procedures identified in the 2019 General Needs Assessment in vascular surgery in Europe (GNA-2019) and identified facilitators and barriers to SBE implementation in vascular surgery. METHODS: A three round iterative survey was distributed via the European Society for Vascular Surgery and the Union Européenne des Médecins Spécialistes. Members from leading committees and organisations within the European vascular surgical community were invited to participate as key opinion leaders (KOLs). Three online survey rounds explored demographics, SBE availability, and facilitators and barriers to SBE implementation. RESULTS: Overall, 147 KOLs (target population 338) accepted invitation to round 1, representing 30 European countries. The dropout rates for rounds 2 and 3 were 29% and 40%, respectively. Most respondents (88%) were senior, consultant level or higher. No mandatory SBE training was required in their department before training on patients, according to 84% of the KOLs. There was high consensus on the need for structured SBE (87%) and mandatory SBE (81%). SBE is available for the top three prioritised procedures in GNA-2019 (basic open skills, basic endovascular skills, and vascular imaging interpretation) in 24, 23, and 20 of the 30 represented European countries, respectively. The highest ranking facilitators were structured SBE programmes, availability of simulation equipment locally and regionally, good quality simulators, and having a dedicated person running the SBE. The highest ranked barriers were lack of structured SBE curriculum, equipment costs, lack of SBE culture, no or limited dedicated time for faculty to teach in SBE, and clinical work overload. CONCLUSION: Based largely on the opinions of KOLs in vascular surgery in Europe, this study revealed that SBE is needed in vascular surgery training and that systematic and structured programmes are required to ensure successful implementation.

Surgeons' Leadership Style and Team Behavior in the Hybrid Operating Room: Prospective Cohort Study.

OBJECTIVE: This study aimed to assess the relationship between surgeons' leadership style and team behavior in the hybrid operating room through video coding. Secondly, possible fluctuations possible fluctuations in leadership styles and team behavior during operative phases were studied. BACKGROUND: Leadership is recognized as a key component to successful team functioning in high-risk industries. The 'full range of leadership' theory is commonly used to evaluate leadership, marking transformational, transactional, and passive. Few studies have examined the effects of these leadership styles on team behavior in surgery and/or their fluctuations during surgery. METHODS: A single-center study included patients planned for routine endovascular procedures. A medical data capture system was used to allow post hoc video coding through Behavior Anchored Rating Scales. Multilevel statistical analysis was performed to assess possible correlations between leadership style and 3 team behavior indicators (speaking up, knowledge sharing, and collaboration) on an operative phase level. RESULTS: Twenty-two cases were analyzed (47 hours recording). Transformational leadership is positively related to the extent to which team members work together (γ=0.20, P <0.001), share knowledge (γ=0.45, P <0.001), and speak up (γ=0.64, P <0.001). Passive leadership is significantly positively correlated with speaking up (γ=0.29, P =0.004). Leadership style and team behavior clearly fluctuate during a procedure, with similar patterns across different types of endovascular procedures. CONCLUSIONS: Consistent with other professional fields, surgeons' transformational leadership enhances team behavior, especially during the most complex operative phases. This suggests that encouraging surgeons to learn and actively implement a transformational leadership style is meaningful to enhance patient safety and team performance.

Assessing endovascular team performances in a hybrid room using the Black Box system: a prospective cohort study.

BACKGROUND: The hybrid room (HR) is a complex, high-risk environment, requiring teams (surgeons, anesthesiologists, nurses, technologists) to master various skills, including the 'As Low As Reasonably Achievable' principle of radiation safety. This prospective single center cohort reports the first use of the Operating Room Black Box (ORBB) in a HR. This medical data recording system captures procedural and audio-visual data to facilitate structured team performance analysis. METHODS: Patients planned for endovascular repair of an infrarenal abdominal aortic aneurysm (EVAR) or treatment of symptomatic iliac-femoral-popliteal atherosclerotic disease (Peripheral Vascular Interventions or PVI) were included. Validated measures and established assessment tools were used to assess (non-)technical skills, radiation safety performance and environmental distractions. RESULTS: Six EVAR and sixteen PVI procedures were captured. Technical performance for one EVAR was rated 19/35 on the procedure-specific scale, below the 'acceptable' score of 21. Technical skills were rated above acceptable in all PVI procedures. Shared decision making and leadership were rated highly in 12/22 cases, whereas surgical communication and nurses' task management were rated low in 14/22 cases. Team members rarely stepped back from the C-arm during digital subtraction angiography. Radiation safety behavior was scored below 'acceptable' in 14/22 cases. A median (interquartile range) number of 12 (6-23) auditory distractions was observed per procedure. CONCLUSIONS: The ORBB facilitates holistic workplace-based assessment of endovascular performance in a HR by combining objective assessment parameters and rating scale-based evaluations. Strengths and weaknesses were identified in team members' (non-)technical and radiation safety practices. This technology has the potential to improve vascular surgical practice, though human input remains crucial. (NCT04854278).

Step, Error, and Event Frameworks in Endovascular Aortic Repair.

OBJECTIVE: Competency-based surgical education requires detailed and actionable feedback to ensure adequate and efficient skill development. Comprehensive operative capture systems such as the Operating Room Black Box (ORBB; Surgical Safety Technologies, Inc), which continuously records and synchronizes multiple sources of intraoperative data, have recently been integrated into hybrid rooms to provide targeted feedback to endovascular teams. The objective of this study is to develop step, error, and event frameworks to evaluate technical performance in elective endovascular aortic repair (EVAR) comprehensively captured by the ORBB (Surgical Safety Technologies, Inc; Toronto, Canada). METHODS: This study is based upon a modified Delphi consensus process to create evaluation frameworks for steps, errors, and events in EVAR. International experts from Vascular Surgery and Interventional Radiology were identified, based on their records of publications and invited presentations, or serving on relevant journal editorial boards. In an initial open-ended survey round, experts were asked to volunteer a comprehensive list of steps, errors, and events for a standard EVAR of an infrarenal aorto-iliac aneurysm (AAA). In subsequent survey rounds, the identified items were presented to the expert panel to rate on a 5-point Likert scale. Delphi survey rounds were repeated until the process reached consensus with a predefined agreement threshold (Cronbach α>0.7). The final frameworks were constructed with items achieving an agreement (responses of 4 or 5) from greater than 70% of experts. RESULTS: Of 98 invited proceduralists, 38 formed the expert consensus panel (39%), consisting of 29 vascular surgeons and 9 interventional radiologists, with 34% from North America and 66% from Europe. Consensus criteria were met following the third round of the Delphi consensus process (Cronbach α=0.82-0.93). There were 15, 32, and 25 items in the error, step, and event frameworks, respectively (within-item agreement=74%-100%). CONCLUSION: A detailed evaluation tool for the procedural steps, errors, and events in infrarenal EVAR was developed. This tool will be validated on recorded procedures in future work: It may focus skill development on common errors and hazardous steps. This tool might be used to provide high-quality feedback on technical performance of trainees and experienced surgeons alike, thus promoting surgical mastery.

International Implementation of a PROficiency based StePwise Endovascular Curricular Training (PROSPECT) in Daily Practice.

OBJECTIVE: A randomised controlled trial (RCT) showed superior real life endovascular performance of trainees after PROficiency based StePwise Endovascular Curricular Training (PROSPECT) compared with traditional training. This study aimed to determine whether PROSPECT can be implemented routinely in surgical training (registry) and to identify the associated effects, facilitators, and barriers to implementation. METHODS: This was a multicentre prospective registry to evaluate real life PROSPECT implementation effects. Cognitive ability was tested with a Multiple Choice Question test (MCQ) (max. score 20 points), while technical skills were assessed with a Global Rating Scale (GRS) (max. score 55 points), Examiner's Checklist (max, score 85 points), and simulator metrics. A cross sectional anonymous survey for both trainees and faculty evaluated the programme's implementation. RESULTS: The PROSPECT registry was implemented in five centres in four countries over a period of three years. Only 17 of 48 starting trainees completed PROSPECT, resulting in a dropout rate of 65%. Participants who completed the programme showed significant improvement in the GRS (median 26 vs. 44, p = .018) and Examiner's Checklist (median 53.5 vs. 80, p = .028) after the programme. The survey was completed by 13/17 faculty members (76%) and 38/57 trainees who participated either in the initial PROSPECT RCT or the registry (67%). PROSPECT was perceived to provide more endovascular training opportunities, but non-availability of logistical/technical support, lack of training time within the working schedule, and little trainee motivation hindered broad implementation. CONCLUSION: PROSPECT has a significant effect on performance, and trainees and faculty agree that PROSPECT is a valuable addition to surgical training. However, external and internal factors pose significant barriers to integration of this simulation based programme into daily practice leading to a high dropout rate. To integrate PROSPECT into contemporary surgical training, it should be a prerequisite to treating real patients and protected training time should be provided, combined with accessible support.

Introduction of a surgical Black Box system in a hybrid angiosuite: Challenges and opportunities.

PURPOSE: In laparoscopy, the Operating Room Black Box® (ORBB) provides insights into operative performance to improve patient safety. This technology may also enhance endovascular surgical practice; however, the use of a C-arm and X-rays pose important challenges, hindering transferability to an endovascular context. We describe the first implementation of ORBB technology in a hybrid angiosuite and illustrate its value in evaluating surgeons' radiation safety, technical and non-technical performance. METHODS: Team members (surgeons, nurses, anaesthesiologists) and stakeholders were informed during several information sessions. Together with teams from Surgical Safety Technologies (Toronto, Canada), an implementation plan was developed, and video-evaluation frameworks were chosen. Radiation safety was assessed using dose measurements and video-evaluation of safety-related behaviours. Technical performance was assessed using 'global' (GRS) and 'procedure-specific' (PRS) rating scales and the 'EndoVascular Aortic Repair Assessment of Technical Expertise' (EVARATE) framework. Surgeons' non-technical skills were assessed with the NOTSS framework. RESULTS: The system captures Audio-visual data from four ceiling-mounted cameras, three ceiling-array microphones, the fluoroscopy screen and anaesthesia monitor. After patient and team consent, an elective endovascular aneurysm repair was successfully analysed. Dose-Area-Product and Air Kerma were 71094 mGy.cm2 and 270 mGy, respectively. Behavioural analysis revealed deficiencies in stepping back and radiation safety communication. Technical skill assessment was feasible: GRS: 29/40; 'PRS': 27/35; EVARATE: 29/35. Non-technical analysis highlighted surgeons' leadership qualities. CONCLUSION: An innovative data capture platform has been successfully installed to evaluate overall performance during endovascular procedures. This technology may facilitate identification of (radiation) safety-related errors and instigate educational interventions based on real-world issues.

Team perception of the radiation safety climate in the hybrid angiography suite: A cross-sectional study.

BACKGROUND: Good radiation safety practice in the angiosuite is essential to protect patients and healthcare workers. Most strategies aim to advance radiation safety through technological upgrades and educational initiatives. However, safety literature suggests that additional ways to improve radiation safety in the angiosuite do exist. The safety climate reflects the way team members perceive various key characteristics of their work environment and is closely related to relevant safety outcomes. A specific 'radiation safety climate' has not been described nor studied in the hybrid angiosuite. This study explores the radiation safety climate in the hybrid angiosuite and its relation to team members' radiation safety behavior, knowledge and motivation. MATERIALS AND METHODS: Vascular surgeons, fellows/trainees and operating room nurses active in the angiosuite at five hospitals were invited to complete an online self-report questionnaire assessing the radiation safety climate (28 items); radiation safety behavior; radiation safety knowledge and radiation safety motivation. Relations between climate scores and behavior were investigated using Pearson correlations. Mediation was analyzed using the Baron and Kenny analysis. P-Values < 0.05 were considered statistically significant. RESULTS: No major differences were identified in total radiation safety climate scores between centers or team member functions. Scale reliability for radiation safety climate was good to excellent (α > 0.663). Total radiation safety climate scores were positively related to the radiation safety behavior score (r = 0.403; p = 0.015). This relation was partially mediated by radiation safety knowledge (ß = 0.1730; 95% CI: [0.0475; 0.3512]), while radiation safety motivation did not act as a mediator: (ß = 0.010; 95% CI: [-0.0561; 0.0998]). CONCLUSION: A well-developed radiation safety climate in the hybrid angiosuite fosters positive radiation safety behaviors, which may partially be explained through improved radiation safety knowledge transfer. Further research on (radiation) safety climate and its impact on radiation safety-related outcome measures for patients is recommended.

Evolution of surgical treatment of carotid artery stenosis: a single center observational study.

Background: In 2009 and 2011 respectively ESVS and AHA/ASA guidelines recommended to operate patients with a symptomatic carotid artery stenosis within 14 days. This study aimed primarily to determine if an academic hospital has implemented these international guidelines about indication and timing of surgical treatment of carotid stenosis. Second, the influence of referral from another hospital on time from symptoms to surgery and the influence of time between neurological event and surgery on 30-day complication rate was studied. Third, the number of asymptomatic carotid artery lesions treated surgically was also evaluated in both periods.Methods: Retrospective study to compare patients with significant atherosclerotic carotid stenosis who underwent carotid endarterectomy (CEA) or carotid artery stenting (CAS) in 2005-2006 versus patients treated in 2014-2016. Demographic data, treatment characteristics, interval between symptom and surgery and 30-day outcomes were collected.Results: In 2005-2006 38.1% (59/155) of the patients were treated for symptomatic carotid artery stenosis, in 2014-2016 this increased to 66.5% (121/182) (p < .001, 95% CI: 0.179-0.383). Median time from neurological symptom to surgery in symptomatic patients decreased from 30 to 13 d (p <.001, 95% CI: 1.476-2.763). Early surgery did not increase the 30-day postoperative complications (p = .19, 95% CI: 0.987-1.003). Referral from another hospital almost doubled the time interval between symptoms and surgery in 2014-2016 (p <.001, 95% CI: 1.386-2.827).Conclusions: Since the publication of the international guidelines, patients with symptomatic carotid artery stenosis were preferably surgically treated within 2 weeks at an academic institution. The number of treated asymptomatic carotid stenoses was drastically reduced.